A Realistic Approach Towards Users' Simulation

Simulation has been proposed and utilised widely in the field of the evaluation of information retrieval (IR) and interactive IR (IIR) systems. It can significantly reduce costs, make experiments easier to reproduce and save time to users and researchers. The question of how realistic these simulations are remains, to a great extent, unexplored. This is due to the fact that searching for information is a self-directed activity, and varies among users in terms of their information seeking behaviours (ISB) and their relevance judgments. Such variations are affected by a number of attributes describing users, tasks, and systems and their interactions. By identifying these attributes researchers could design more effective user models and realistic simulations. This paper presents a user-centric evaluation methodology based on user profiles and ISBs.